Method and apparatus for calibrating a tiled display

ABSTRACT

A display system that can be calibrated and re-calibrated with a minimal amount of manual intervention. To accomplish this, one or more cameras are provided to capture an image of the display screen. The resulting captured image is processed to identify any non-desirable characteristics, including visible artifacts such as seams, bands, rings, etc. Once the non-desirable characteristics are identified, an appropriate transformation function is determined. The transformation function is used to pre-warp the input video signal that is provided to the display such that the non-desirable characteristics are reduced or eliminated from the display. The transformation function preferably compensates for spatial non-uniformity, color non-uniformity, luminance non-uniformity, and other visible artifacts.

CROSS REFERENCE TO CO-PENDING APPLICATIONS

The present application is related to U.S. patent application Ser. No.09/159,340, filed Sep. 23, 1998, entitled “METHOD AND APPARATUS FORPROVIDING A SEAMLESS TILED DISPLAY”, and U.S. patent application Ser.No. 09/159,024, filed Sep. 23, 1998, entitled “METHOD AND APPARATUS FORCALIBRATING A DISPLAY USING AN ARRAY OF CAMERAS”, both of which areassigned to the assignee of the present invention and incorporatedherein by reference.

“The invention described herein was made in the performance of workunder NASA Contract NAS1-20219 and is subject to the provisions ofSection 305 of the National Aeronautics and Space Act of 1958, asamended (42 USC 2457).”

BACKGROUND OF THE INVENTION

This invention relates to calibrating displays, and more particularly,to calibrating tiled projection displays that use multiple projectors toproduce larger and/or higher resolution images.

Multiple projector systems have been proposed and used for many years.In the 1950s, the “CINERAMA” system was developed for the film industry.The CINERAMA system projected three films using three separateprojectors, which were combined to form a single panoramic image.Disneyland continues to use a similar multiple projector system. AtDisneyland, a circle of projectors shines onto a screen that circles thewall of a round room.

In the video field, multiple projector systems have been proposed andused for a number of specialty applications. For example, U.S. Pat. No.4,103,435 to Herndon and U.S. Pat. No. 3,833,764 to Taylor suggest usingmultiple projector systems for flight simulators. In many of thesesystems, multiple video screens are placed next to each other to form alarge image display for multiple projectors. A difficulty with many ofthe video based multiple projector display systems is making themultiple images appear as one single continuous image on the displayscreen.

When two images are projected side-by-side on a single screen, there isnormally a seam between the images. The final display image will eitherappear as two images placed side-by-side with a gap in between or, ifthe images are made to overlap on a single screen, there will be abright line where the two images overlap. Because of the inconsistenciesin conventional cameras, video processing, delivery channels, displaysand, specifically, projectors, it is exceedingly difficult to perfectlymatch the resultant video images so that no tiling artifacts appearamong the images. If the images are brought very close together on thesame screen, there is typically both gaps and overlaps at each seam.

The article entitled Design Considerations and Applications forInnovative Display Options Using Projector Arrays, by Theo Mayer, SPIEVol. 2650 (1996), pp. 131-139, discloses projecting a number of discreteimages in an overlapping relation and ramping the brightness of thediscrete images in the overlapping regions of each image. Mayerdiscloses using a blending function to fade down each overlapping edgeof the discrete images in such a way so as to compensate for the gamma(video signal reduction vs. light output curve) of a phosphor, lightvalve or LCD projector, with the goal of producing a uniform brightnessacross the display.

U.S. Pat. No. 5,136,390 to Inova et al. recognizes that the blendingfunction typically cannot be a simple even ramping function. A typicalvideo projector produces an image that becomes darker toward the edgesof the image as a natural function of the lens system used, and has anumber of bright and dark portions caused by normal irregularities inthe signal, intermediate signal processor, projector, screen, etc. Theseinconsistencies typically vary from one video component to another, andeven among different components with similar construction. Also,different types of projectors often respond differently to the sameamount of brightness modification. Thus, a simple ramp of the brightnessin the over-lapping regions can produced light and dark bands and/orspots in the resulting image.

To overcome these limitations, Inova et al. suggest applying a simpleeven blending function to the overlapping regions of the image, assuggested by Mayer, but then manually tuning the simple even blendingfunction at specific locations to remove the visible artifacts from thedisplay. The location of each artifact is identified by manually movinga cursor over each location that is identified as having an artifact.Once the cursor is in place, the system tunes the corresponding locationof the blending function so that the corresponding artifacts areremoved.

Since each artifact must be manually identified by a user, the processof calibrating an entire display can be time consuming and tedious. Thisis particularly true since many displays require periodic re-calibrationbecause the performance of their projectors and/or other hardwareelements tend to change over time. In view of the foregoing, it would bedesirable to have a display that can be calibrated and re-calibratedwith less manual intervention than is required by Inova et al. andothers.

SUMMARY OF THE INVENTION

The present invention overcomes many of the disadvantages of the priorart by providing a display that can be calibrated and re-calibrated withlittle or no manual intervention. To accomplish this, the presentinvention provides one or more cameras to capture an image on thedisplay screen. The resulting captured image is processed to identifyany non-desirable characteristics including visible artifacts such asseams, bands, rings, etc. Once the non-desirable characteristics areidentified, an appropriate transformation function is determined. Thetransformation function is used to pre-warp the input video signal suchthat the non-desirable characteristics are reduced or eliminated fromthe display. The transformation function preferably compensates forspatial nonuniformity, color non-uniformity, luminance non-uniformity,and/or other visible artifacts.

In one illustrative embodiment, a tiled projection display is providedthat has two or more projectors arranged in an array configuration. Theprojectors may be direct write (e.g. CRT, LCD, DMD, CMOS-LCD) or anyother type of projector. In a tiled type display, each of the projectorspreferably projects a discrete image separately onto a screen, whereinthe discrete images collectively form a composite image. The discreteimages may or may not overlap one another. A camera is then directed atthe screen to capture a capture image of at least a portion of thecomposite image. The capture image may encompass less than one tile,about one tile, the entire composite image, or any other portion of thecomposite image that is deemed desirable.

A determining block then determines if the capture image has one or morenon-desirable characteristics. The non-desirable characteristics may bedetermined by comparing the capture image, or a portion thereof, with apredetermined data set as more fully described below. The determiningblock is preferably provided in a processor or the like. In oneillustrative embodiment, the processor resides in one location andservices all projectors. In another illustrative embodiment, theprocessor function is physically distributed among the projectors.

Once the non-desirable characteristics are determined, an identifyingblock identifies a transformation function that can be used to processthe input video signal and provide processed input video signals toselected projectors to reduce the non-desirable characteristics in thecomposite image. The non-desirable characteristics may include spatialnon-uniformity, color non-uniformity, and/or luminance non-uniformity,but may also include other known image artifacts or irregularities.

To determine the spatial distortion of the projection display, an inputsignal may be provided to selected projectors to project a number ofdiscrete images, each exhibiting a predetermined pattern. The cameradevice can then be used to capture a capture image of at least a portionof the screen. Using the capture image, the distortion of the projectiondisplay can be determined by, for example, comparing the capture imagewith a predetermined expected image. Alternatively, or in addition to,the distortion can be determined by comparing the location of selectedfeatures of the predetermined pattern in adjacent discrete images, andmore preferably, in selected overlapping regions between images. Byusing an affine, perspective, bilinear, polynomial, piecewisepolynomial, global spline, or similar technique, a transformationunction can be determined and applied to the input video signal tocompensate for the spatial distortion of each projector.

To determine the color and luminance distortion of the projectionsystem, a number of input signals of varying intensity may besequentially input to the projection display, wherein each input signalcorresponds to a flat field image of a selected color. For example, afirst input signal may correspond to a red flat field image having anLCD intensity of “255”. The next input signal may also correspond to ared flat field image, but may have a LCD intensity of “220”. Inputsignals having progressively lower intensity may be provided until theinput signal has a LCD intensity of “0”. This process may be repeatedfor both blue and green flat field images. The camera device preferablycaptures each of the flat field images, either as a single image if thefield-of-view of the camera device corresponds to the entire display, oras multiple images if the camera device has a smaller field-of-view. Theresulting images are preferably stored as an array of capture images.Once collected, the non-desirable characteristics of each capture imagecan be determined including the luminance domes for each projector.Thereafter, a transformation function may be determined for reducing theluminance domes across selected tiles, and matching the brightness andcolor of each tile with adjacent tiles.

It is contemplated that the camera device may be, periodically activatedto capture a new capture image. The determining block may then determineif the newly captured image has one or more non-desirablecharacteristics, as described above, and the identifying block mayidentify a new transformation function that can be used to process theinput video signal and provide processed input video signals to selectedprojectors to reduce the identified non-desirable characteristics. Thus,it is contemplated that the present invention may also be used toperiodically re-calibrate the display with little or no manualintervention.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects of the present invention and many of the attendantadvantages of the present invention will be readily appreciated as thesame becomes better understood by reference to the following detaileddescription when considered in connection with the accompanyingdrawings, in which like reference numerals designate like partsthroughout the figures thereof and wherein:

FIG. 1 is a perspective view of a four-by-six array of projectors;

FIG. 2 is a perspective view of one illustrative projector of FIG. 1;

FIG. 3 is a schematic diagram of an illustrative embodiment of thepresent invention with the field-of-view of the camera encompassing twoor more tiles;

FIG. 4 is a block diagram showing an illustrative implementation for theprocessor block of FIG. 3;

FIG. 5 is a schematic diagram of an embodiment similar to that shown inFIG. 3, but with the field-of-view of the camera encompassing only aboutone tile;

FIG. 6 is a schematic diagram of an embodiment similar to that shown inFIG. 3, but with the processing function of FIG. 3 distributed among theprojectors;

FIG. 7 is block diagram showing another embodiment of the presentinvention;

FIG. 8 is a flow diagram showing an illustrative method for calibratinga display;

FIG. 9 is a flow diagram showing another illustrative method forcalibrating a display, and in particular, a tiled display;

FIG. 10 is a flow diagram showing yet another illustrative method forcalibrating a display, including distinguishing the distortionintroduced by the camera from the distortion introduced by the rest ofthe display;

FIG. 11 is a diagram showing an illustrative pattern that is displayedand later captured for determining spatial distortions in the display;

FIG. 12 is a diagram showing the illustrative pattern of FIG. 11displayed on two adjacent and overlapping tiles, also for determiningspatial distortions in the display;

FIG. 13 is a diagram showing the operation of an illustrativetransformation function that can be used to reduce the spatialdistortion in a display by moving selected features toward a correctivelocation;

FIG. 14 is a diagram showing the operation of an illustrativetransformation function that may be used to reduce the spatialdistortion in a display by moving selected features toward a correctivelocation by a distance that is related to a relative method, a weightedaverage for example, modified by composite image or global constraints;

FIG. 15 is a flow diagram showing an illustrative method for at leastpartially removing a spatial distortion from the display;

FIG. 16 is a flow diagram showing an illustrative method for identifyinga transformation for a tiled display to at least partially removing aspatial distortion from the tiled display;

FIG. 17 is a graph showing the luminance domes for a LCD projector atvarious input intensities and showing how the dome shapes changedepending on the input intensity level;

FIG. 18 is a schematic diagram showing the luminance domes for threetiled LCD projectors each at various input intensities; and

FIG. 19 is a flow diagram showing an illustrative method for at leastpartially removing a luminance distortion from the display.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention provides a tiled display that can be calibratedand recalibrated with a minimal amount of manual intervention. Toaccomplish this, the present invention provides one or more cameras tocapture an image of the display screen. The resulting captured image isprocessed to identify any non-desirable characteristics includingvisible artifacts such as seams, bands, rings, etc. Once thenon-desirable characteristics are identified, an appropriate transformation function is determined. The transformation function is us edto pre-warp the input video signal such that the nondesirablecharacteristics are reduced or eliminated from the display. Thetransformation function preferably compensates for spatialnon-uniformity, color non-uniformity, luminance non-uniformity, andother visible artifacts.

In one illustrative embodiment, a tiled display is provided that has twoor more projectors arranged in an array configuration. The displays maybe projection displays which use CRT, LCD, DMD, CMOS-LCD or any othertype of imaging device, and may be front or rear projection types. In atiled type display, each of the projectors preferably images or projectsa discrete image separately onto a surface or screen, wherein thediscrete images collectively form a composite image. The discrete imagesmay or may not overlap one another. Such a configuration is shown inFIG. 1.

An illustrative projector 8 is shown in FIG. 2, and preferably uses oneDigital Micromirror Device (DMD) 10. DMD devices typically include anarray of electronically addressable, movable square mirrors that can beelectrostatically deflected to reflect light. The use of a DMD devicecan provide a lightweight, reliable, digital display with a wide viewingangle and good picture clarity. Further, some DMD devices meet variousMIL-STD-810 environmental and stress requirements, and can display colorgraphic, text and video data at various frame rates.

The projector 8 also preferably includes various optical elements toproperly prepare the incoming illuminations, illuminate the DMD 10, andproject the outgoing image. As shown in FIG. 2, the optical path mayinclude two segments: the illumination path 12 and the projection path14. The optical path may start with a high reliability, metal halide,short-arc lamp 16 that illuminates the DMD 10. The light from the arclamp 16 passes through a rotating RGB color filter wheel 18. Anillumination relay lens magnifies the beam to illuminate the DMD 10 andform a telecentric image at the DMD 10. A Total Internal Reflection(TIR) prism 20 enables the incoming light from the lamp to pass onto theDMD 10, and back into the projection optics. Depending on the rotationalstate (e.g. ±10 degrees for on/off) of each mirror on the DMD, the lightfrom the DMD 10 is directed into the pupil of the projection lens (on)or away from the pupil of the projection lens (off). A multiple-elementprojection cell magnifies the image coming off the DMD 10, at thedesired MTF, lateral color, and distortion.

Each projector 8 may also include an electronics module (not explicitlyshown). The electronics module may take the incoming data signals,convert the temporal signals into spatial representations on the DMD 10,and control the filter 18 that provides the sequential color for thedisplay. As described below, the electronics may be modular, allowing anarbitrary number of projectors to be tiled together. Further, tilingalgorithms may be incorporated into the electronics, as appropriate, toenable “smart” projectors. This may allow the electronics of eachprojector to automatically or manually adapt to an arbitraryconfiguration of projectors, with little or no manual intervention bythe user.

FIG. 3 is a schematic diagram of an illustrative embodiment of thepresent invention with the field-of-view of the camera encompassing twoor more tiles. The system is generally shown at 50, and includes aprocessor 52, a first display which may be a projector 54, a seconddisplay which may be a projector 56, a viewing surface or a screen 58and a camera 62. For the purpose of illustration the display will bediscussed as a projector. The first and second projectors each project adiscrete image onto the screen 58, as shown. The discrete images may beoverlapping or non-overlapping, and may form a composite image on thescreen 58.

The processor 52 receives an input video stream 66. Because each of theprojectors 54 and 56 project a portion of the desired composite image,the processor 52 preferably segments the input video stream 66 into afirst input video signal 72 and a second input video signal 74. In theillustrative embodiment, the processor 52 segments the input videostream so that any overlaps between adjacent discrete images, forexample overlap 60, are taken into account as is known in the art.

The input video stream 66 may be provided from any number of sources,and may be a NTSC, PAL, HDTV, workstation or PC video signal. Thesesignal types are compatible with the RS-170 or RS-343 guidelines andspecifications, for example, or more recently the VESA video signalstandards and guidelines. The signals may include horizontal andvertical sync, and blanking information in addition to the active videosignal used to build the output image. The sync signals may be used bythe processor 52 to derive a system and/or video-sampling clock,especially in the case of an analog input signal that needs to bedigitized.

Camera 62 is directed at the screen 58 as shown, and provides a cameraoutput signal to the processor 52 via interface 64. The camera 62 mayhave a field-of-view that is sufficient to capture a capture image of atleast a portion of the composite image. It is contemplated that thefield-of-view may encompass only 1% of the composite image, 50% of thecomposite image, the entire composite image, or any other portion of thecomposite image that is deemed desirable. In a tiled display, this maycorrespond to only a portion of one tile, more than one tile, or all ofthe tiles. When the field-of-view of the camera does not encompass theentire display, it may be necessary to capture a capture image of eachsection of the display separately, and then assemble the results in abackground or real-time mode to achieve a calibrated display over allthe tiles.

In the embodiment shown, the camera 62 has a field-of-view that issufficient to encompass the discrete images provided by the firstprojector 54 and the second projector 56. The capture image is providedto the processor 52 as a feedback image via interface 64. A determiningblock, provided in processor 52, may determine if the capture image hasone or more non-desirable characteristics. Preferably, the non-desirablecharacteristics are determined by comparing the capture image, or aportion thereof, with a predetermined data and information set as morefully described below.

Once the non-desirable characteristics are determined, an identifyingblock, preferably within the processor 52, identifies a transformationfunction that can be used to process the input video stream 66 andprovide processed input video signals to projectors 54 and 56 whichreduce the non-desirable characteristics in the composite image. Thenon-desirable characteristics may include spatial non-uniformity, colornon-uniformity, and/or luminance non-uniformity, but may also includeother known image artifacts or irregularities.

It is contemplated that the projection display may be a front or rearprojection display, and the camera 62 may be positioned in front ofand/or behind the screen. In a second illustrative embodiment, the tiledprojection display is a rear projection display having an array of tiledLCD type projectors, with each projector projecting a discrete imageonto the back side of a transmissive screen 58. The transmissive screen58 is viewed from the font side, and the camera 62 is used to capture acapture image of at least a portion of the screen 58 from the frontside.

It is contemplated that the camera 62 may be a still or video electroniccamera, or have an equivalent combination of components that capture thescene in a multi-point manner and deliver an electronic representationof the image to the Processor 52. In the preferred embodiment, thecamera 62 is a CCD or CMOS; camera, either color (e.g. multipointcalorimeter) or monochrome. The camera 62 preferably includes a photopicfilter to enable the camera 62 to measure the output image in a mannerthat is consistent with the human visual system. Thus, noise and errorsin luminance and chromaticity are measured in a way that is similar tohow the eye detects such anomalies. The image may be a snapshot takenover a brief moment (e.g. less than 60 milliseconds), or over a longerexposure time (e.g. on the order of one second).

In a preferred embodiment, the camera 62 may be a conventional cameradevice, such as a video miniature camera that produces an analog output.The analog output is digitized and captured by a frame grabber or thelike located in the processor 52. Once digitized the capture image canbe stored and processed using digital processing techniques. Todetermine if the capture image has any non-desirable characteristics,the capture image may be compared to a predetermined data or informationset. First, however, the distortion introduced by the camera 62 andassociated processing hardware may be determined and removed.

To isolate the camera distortion, it is contemplated that a physicaltemplate 68 may be provided in front of the screen 58, as shown. Thephysical template 68 preferably includes a predetermined patternthereon, such as an array of dots. With the physical template 68 inplace, the camera 62 may capture a capture image of at least a portionof the physical template 68 including a portion of the predeterminedpattern. By comparing the capture image with a predetermined expectedimage,. and in particular, comparing the location of the dots of thepredetermined pattern in the capture image to the expected locations ofeach of the dots, the distortion of the camera and associated hardwarecan be determined. Using the deviation from the expected locations, atransformation function can be determined and applied to the input videostream 66 to compensate for the camera distortion.

After the camera distortion is determined, the physical template 68 maybe removed, and the distortion of the display itself can be determined.The display may have a number of types of distortion including spatialdistortion, color distortion, luminance distortion, etc. To determinethe spatial distortion of the projection display, for example, an inputsignal may be provided to selected projectors 54 and 56 to project anumber of discrete images, each exhibiting a predetermined or knownpattern. The camera 62 can then be used to capture a capture image of atleast a portion of the screen 58. Using the capture image, thedistortion of the projection display can be determined by, for example,comparing the capture image with a predetermined and/or expected image.Alternatively, or in addition to, the distortion can be determined bycomparing the location of selected features of the predetermined patternin adjacent discrete images, and more preferably, in selectedoverlapping regions 60 between images. By using an affine, perspective,bilinear, polynomial, piecewise polynomial, global spline or similartechnique, a transformation function can be determined and applied tothe input video stream 66 to compensate for the spatial distortion ofthe projectors 54 and 56. Preferably, the distortion introduced by thecamera 62 is removed from the capture image, as described above, beforethe distortion of the projection system is determined.

To determine the color and luminance distortion of the projectionsystem, a number of input signals of varying intensity may besequentially input to the projection display, wherein each input signalcorresponds to a flat field image of a selected color. For example, afirst input signal may correspond to a red flat field image having anLCD input intensity of “255” or the brightest input value. The nextinput signal may also correspond to a red flat field image, but may havea dimmer LCD input intensity of “220”. Input signals havingprogressively lower intensity may be provided until the input signal hasa LCD input intensity of “0” or black, the dimmest input value. Theseinputs may be expressed as bright to dark equivalents especially if theinput is an analog voltage instead of a digitally measured value. Thisprocess may be repeated for both blue and green flat field images. Thecamera 62 preferably captures each of the flat field images, either as asingle image snapshot taken periodically when the field-of-view of thecamera 62 corresponds to the entire display, or as multiple images ifthe camera device has a smaller field-of-view. The resulting images arepreferably stored as an array of capture images or compressed versionsthereof in a memory within processor block 52. Once collected, thenon-desirable characteristics of each capture image can be determinedincluding the color corresponding and input intensity variant luminancedomes of each projector 54 and 56.

Once the luminance domes are identified, a ceiling and floor may bedetermined for both color and intensity, across the entire display. Forexample, one projector may be brighter than another even though all aredriven at a maximum intensity (e.g. LCD “255”), and the brightnessprovided by each projector may decrease near the edges of the image.Accordingly, a ceiling may be selected to match the dimmestsuper-position result of all the tiles when all projectors are operatedat maximum intensity. Likewise, a floor may be selected to match thebrightest superposition result when all projectors are operated atminimum intensity (LCD “0”).

Thereafter, a transformation function may be determined for reducing theluminance domes across selected tiles, and for matching the brightnessand color of each tile with adjacent tiles. For example, thetransformation function may be represented by a color look up table ofcaptured or compressed color domes, a nearest neighbor detection andidentification function and an interpolation function among the nearestneighbors to determine the input level needed at the display to outputthe desired linear output level.

In one embodiment, the transformation function makes the luminancevariation across the entire display less than about two percent, whichis less than one just-noticeable-difference (JND) according to Weber'sLaw. To help achieve this level of luminance uniformity, thetransformation function is preferably a function of the X and Y locationon the display, and for some image source technologies such aspolysilicon LCDs, the LCD input intensity value. Preferably, thevariations across the display are held to be less than one JND inaccordance with the contrast modulation sensitivity curve of humanvision. This curve allows more or less variation as a function ofspatial frequency.

When the display has overlapping tiles, it is contemplated that thedistortion of the system may be directly determined from patternsprojected on the display. For a tiled display having overlappingdiscrete images, a first feature may be identified in a selectedoverlapping region, wherein the first feature is projected by a firstprojector. Then, a second feature may be identified in the same selectedoverlapping region, wherein the second feature is projected by a secondprojector, and wherein the second feature corresponding to the firstfeature. The spatial relationship between the first and second featuresmay then be determined, and a first transformation function for thefirst projector can be identified therefrom. Likewise, a secondtransformation function for the second projector can be identified. Afurther discussion of this can be found below with reference to FIGS.12-14.

Finally, it is contemplated that the camera 62 may be periodicallyactivated to capture a new capture image. The determining block inprocessor 52 may determine if the newly captured image has one or morenon-desirable characteristics, and the identifying block of theprocessor 52 may identify a new transformation function that can be usedto process the input video stream 66 and provide processed input videosignals to projectors 54 and 56 to reduce the identified non-desirablecharacteristics. Thus, it is contemplated that the present invention maybe used to periodically re-calibrate the display with little or nomanual intervention. The period of re-calibration may be increased ordecreased as required by the operational environment. For example, itmay be done at a 60 Hz rate to negate effects in a high vibrationenvironment. In a benign environment, such as may happen in a home, theperiod may be reduced to 0.001 Hz or less.

It is also contemplated that processor 52 may include built-in-testlogic. The built-in-self test logic may periodically detect if anyportion of the display has failed, and if so, correcting for the failureby appropriately re-calibrating the display system. This is particularlyuseful when the discrete images overlap one another by about 50 percentor more. The 50% value, as an example, demarcates a packing arrangementwhich is fully redundant, leading to significant fail-operational systemattributes. Fail operational means that a component can fail but thesystem continues to be fully operational. With a 50% overlap, if oneprojector fails, at least one more is ready to fill in the voidresulting in significant gains in system reliability.

To save memory costs, the transformation functions, and the extractedfeatures, information and data sets as described herein, are preferablyrepresented and stored as a number of reduced information sets such asaffine transformation or forward differencing coefficients orcompression coefficients like those recommended in JPEG or MPEGspecifications. Interpolation or the like can then be used toreconstruct the appropriate correction factors for any location amongthe selected points (see FIG. 11 below).

FIG. 4 is a block diagram showing an illustrative implementation for theprocessor block 52 of FIG. 3. In the illustrative embodiment, theprocessor block 52 includes a first sub-processing block 80 forservicing the first projector 54, and a second sub-processing block 82for servicing the second projector 56. An input video segmentor block 84segments the input video stream 66, and provides an appropriatesegmented input video signal to the first sub-processing block 80 andthe second sub-processing block 82.

A Digitizer and Image Memory Block 86 receives the analog video signal64 from the camera 62 and converts it into digital form, typically an 8bit value for red, an 8 bit value for green and another for blue. Theoutput of the camera 62 can be a monochrome signal or color signal. Ifthe output of the camera 62 is monochrome, the test images of separatered, green and blue values may be shown by the projector from time totime and captured separately or in combination by the camera incombination with the Digitizer and Image Memory Block 86. Thedigitization function need not reside within the physical bounds of theprocessor. Rather, it may be a part of the camera itself. The same istrue of the Image Memory. Further, these 88,64 and 86, preferablyimplemented in the apparatus of a CMOS camera, may be embedded in thehardware of the sub-processor block, 80. Further, these may all beembedded in a CMOS-LCD imaging device to achieve the highest level ofintegration.

Because the camera 62 captures an image that corresponds to bothprojectors 54 and 56 (see FIG. 3), a camera segmentor block 88 may beprovided to segment the capture image and provides the appropriateportions thereof to the first and second sub-processing blocks 80 and82.

The first sub-processing block 80 preferably has access to the capturedand ideal images of the first projector 54 and the neighboringprojectors, including the second projector 56. The capture image, or atleast the appropriate portion thereof, is analyzed by the firstsub-processing block 80. For spatial compensation, a number of featuresmay be extracted from the image which may include seeking, detectingidentifying, and extracting anchor points in the image. The anchorpoints may be, for example, features in a predetermined pattern (e.g. anarray of dots) or may be deduced from the standard input video byderiving which features in the input image which are stochasticallyseparable and uniquely identifiable. For color compensation, theprojector(s) under test may project a series of images onto the screenranging in intensity from LCD “0” to LCD “255”, for red, green and blueseparately. The camera 62 may capture a color or monochrome image ofeach of the images on the screen. These capture images are preferablystored as an array in the Reference Images and Data block 90, which isimplemented using storage media. Further, the red, green and bluecompensation information may be obtained at initial setup and adjustedfor example by scaling in real-time or periodically. This means theinput video may be used to alter known detailed compensation data,thereby preferably circumventing the need to apply test images to learnwhat compensation needs to be applied over time and condition.

The vignetting aspects of the camera lens aperture and assembly may alsobe captured and included in the result. The vignette aspect of thecamera can be measured ahead of time using a flat white field imageprovided by a uniformly illuminated white flat field and stored away asa priori information. This information may also be stored in compressedform in the Reference Images and Data block 90.

The reference and measured imagery are compared in the Block InverseTransform Calculator 100. The various test images, including spatial andcolor, are analyzed in this block. Salient and relevant features areextracted automatically preferably using variations of filter,threshold, linearity correction, and gamma correction methods. In orderto obtain spatial compensation, the affine, perspective, bilinear,polynomial, piecewise polynomial, or global spline transformation, forexamples, may be computed by comparing the measured spatial test patternfeatures with resident reference test image features. For colorinformation, the gamma, gain and offsets of the camera, digitizer andprojectors may be extracted. These and related features are categorizedand solved to produce a set of spatial and color compensating transformcoefficients.

The transformation coefficients, calculated typically in a non-real-timemode, are loaded into the Real-time Warper and Color Blender block 102.This block converts the coefficients into high-speed real-timecompensation signals which are provided to the first projector 52. TheTransformed Video Signal 72 is preferably a pre-warped version of theInput Video Stream 66. The pre-warping can be local or global withrespect to the tile and its neighbors. The pre-warping may be applied incolor and space or other artifact dimension, time separation forexample, in a manner that, when the Transformed Video Signal 72 ispassed through the projector-screen system, the output image emerges inspatial-temporal and color alignment, with little or no visibleartifacts.

The Real-time Warper and Color Blender Block 102 can be implementedusing a combination of standard processing components including highspeed look-up tables, high speed digital signal processors, imagememory, X, Y position counters, bilinear interpolation devices and/orforward differencing engines (made from coefficient registers, addersand latches, for example).

It is contemplated that that the alignment may be implemented inrelative or absolute terms. For example, if alignment is done withrespect to a physical template, this may be considered to be an absolutealignment. If, on the other hand, no physical template is used, and thebehavior of the tiles is characterized relative to attributes ofneighbor tiles, then this may be considered a relative alignment.

The second sub-processor block 82 may be constructed in a similarmanner. Because in some applications, the computation of atransformation function for one tile can depend on the information andtransformation function of another tile, it is contemplated that aninterface 106 may be provided between the first and second sub-processorblocks 80 and 82. This interface may allow the first sub-processingblock 80 to communicate with the second sub-processing block. While theprocessor block 52 is shown having two separate sub-processing blocks 80and 82, any number of other implementations are contemplated. Forexample, the processor block 52 may be implemented as an appropriatelyprogrammed general purpose microprocessor, an appropriately programmeddigital signal processor, or my other implementation so long as the sameor similar result is achieved.

FIG. 5 is a schematic diagram of an embodiment similar to that shown inFIG. 3, but in this embodiment, the camera 120 has a field-of-view thatencompasses only about one tile. In this configuration, the camera 120may capture a series of images and its output used to determine atransformation function for the second projector 56, as described above.Subsequently, the camera 120 may be moved or its field of view movedusing, for example, a deflecting mirror so that the field-of-view of thecamera 120 encompasses the discrete image for the first projector 54, asshown at 122. Then, the camera 120 may capture a series of images, asdescribed above, and its output used to determine a transformationfunction for the first projector 54. This may be repeated until atransformation function can be determined for each projector in thedisplay.

FIG. 6 is a schematic diagram of an embodiment similar to that shown inFIG. 3, but with the processing function distributed among theprojectors. As such, the first projector 54 and the second projector 56each have a processor block 130 and 132, respectively, associatedtherewith.

An Inter-Processor I/O is also included. These I/O channels may beimplemented as video channels, parallel and/or serial data bustransmission lines, or any other type of communication link, includingarray position encoders or other array signature means. With the I/Ofunction provided, the processors 130 and 132 form a distributed arrayof processors, potentially eliminating the need for a central executiveprocessor. In one embodiment, processor 130 may assume the function of aglobal executive, processor 132 may assume the function of a color blendcalculator, while another (not shown) may assume the function of aspatial warp calculator, and yet another may assume the function of abuilt in test monitor, etc. Preferably, each processor applies the sametype of appropriate transformation to the corresponding portion of theinput video stream to achieve a real-time transformation process. Whilean executive processor is not precluded, the Inter-Processor I/O 134permits each tile cluster of resources to engage in dialog with itsneighbors. This may be implemented as a local and global arrangement ofinformation including image measurement and system compensation. Thefunction of the processor array may identify the location of each tile,identify the neighboring tiles, and analyze the results includingselected neighboring results. Accordingly, an arbitrary number andconfiguration of tiles may be provided enabling the tiling modules to beadded or subtracted transparently by the user.

FIG. 7 is block diagram showing another embodiment of the presentinvention. An input image signal 140 is provided to a compensationdevice 142, where it is converted to or used as a reference signalimage. Here, features may be extracted from the input image or, in thecase of a calibration template, used as is. There the input signal isrouted through to a video driver block 144. The signal is then providedto the Liquid Crystal Display (LCD) Driver 146. The LCD driver convertsthe input signal into special signals known in the art as required todrive the particular display device. The use of an LCD display is onlyillustrative. It is contemplated that the display device could be a DMD,ferroelectric, CRT or any type of electronic display.

In the example shown, the projector 148 outputs an image signal asmodulated light that provides a viewable image on the screen 10. There,the image is seen by camera 152, which converts the image into anelectronic signal. At each of these stages, distortion may be andtypically is induced in the signal stream. The signal produced by thecamera is then digitized by data translation digitizer block 154,converted into an image representation and compared to the referencesignal image. The comparison is preferably done in terms of spatial andcolor image attributes.

Thereafter, a transformation that corresponds to the distortion of thesystem is generated. To characterize the spatial distortion an 81-pointtest pattern is provided (see, for example, FIGS. 11 and 12). The81-point test pattern is used to generate a set of global transformationcoefficients or different sets of local transform coefficients. Themodels for the spatial distortion of one tile include, for example, theaffine, perspective, bilinear, polynomial, piecewise polynomial, andglobal spline transformations.

In accordance with the above, FIG. 8 shows a flow diagram of anillustrative method for calibrating a display. The algorithm is enteredat element 200, and control is passed to element 202. Element 202captures a capture image of at least a portion of the composite image ona screen. Control is then passed to element 204. Element 204 determinesif the capture image has one or more non-desirable characteristics.Control is then passed to element 206. Element 206 identifies atransformation function that can be used to process an input videosignal and provide a processed input video signal to selected projectorsto reduce the non-desirable characteristics. Control is then passed toelement 208, wherein the algorithm is exited.

FIG. 9 is a flow diagram showing another illustrative method forcalibrating a display, and in particular, a tiled display. The algorithmis entered at element 220, wherein control is passed to element 222.Element 222 segments the input video signal to identify a portion thatcorresponds to each tile of a tiled display. Control is then passed toelement 224. Element 224 selects a first/next tile. Control is thenpassed to element 226. Element 226 applies a transformation to theportion of the input video signal that corresponds to the selectedfirst/next tile by using the coefficients that correspond to thefirst/next tile and, through bi-linear interpolation or the like,producing a corresponding transformed input video signal. Control isthen passed to element 228. Element 228 provides the transformed inputvideo signal to the projector(s) that correspond to the first/next tile.Control is then passed to element 230. Element 230 determined if theselected first/next tile is the last tile in the display. If theselected first/next tile is not the last tile in the display, control ispassed back to element 224. If, however, the selected first/next tile isthe last tile in the display, control is passed to element 232, whereinthe algorithm is existed. While the flow diagram shown in FIG. 9 showsprocessing each of the tiles sequentially, it is contemplated that thetiles may be processed in parallel. FIG. 10 is a flow diagram showingyet another illustrative method for calibrating a display, includingdistinguishing the distortion introduced by the camera from thedistortion introduced by the rest of the display. The algorithm isentered at element 240, wherein control is passed to element 242.Element 242 provides a physical template adjacent to the screen. Thephysical template preferably includes a predetermined pattern. Controlis then passed to element 244. Element 244 captures a capture image ofat least a portion of the physical template using a camera device.Control is then passed to element 246. Element 246 determines a cameradistortion that is introduced by the camera device by comparing thecapture image with a predetermined expectation. Control is then passedto element 248. Element 248 removes the physical template. Control isthen passed to element 250.

Element 250 provides an input signal to selected projectors to project anumber of discrete images, each exhibiting a predetermined pattern. Itis understood that only selected projectors may project a pattern,rather than all projectors. Control is then passed to element 252.Element 252 captures a capture image of at least a portion of the screenusing the camera device. Control is then passed to element 254. Element254 reduces or removes the distortion introduced by the camera from thecapture image. Control is then passed to element 256. Element 256determines a transformation function for reducing or removing thedistortion introduced by the projection system by comparing the captureimage with a predetermined expectation. Control is then passed toelement 258, wherein the algorithm is exited.

FIG. 11 is a diagram showing an illustrative pattern with 9×9 dots thatmay be displayed and later captured for determining spatial distortionsin a display. In the illustrative embodiment, each tile 268 is dividedinto eight segments across and eight segments down resulting in 64quadrilateral regions. The vertices of each region are the correspondingtie points. Accordingly, the tie points in the array of regions are usedto determine the local distortion in region 270 and others across thetile 268. Different sets of local transformation coefficients correspondto different quadrilateral regions. The geometric distortion within eachregion is modeled by a transformation function governed by a pair ofbilinear equation with eight degrees of freedom. The eighttransformation coefficients are determined by comparing the knownlocations of the four tie points in the capture image to thecorresponding expected locations as determined, for example, using acapture image of the template overlay.

The appropriate correction factor for those locations that fall betweenthe dots (for example, location 272) can be determined by using bilinearinterpolation or the like. A further discussion of bilineartransformations can be found in Digital Image Warping, by GeorgeWolberg, IEEE Computer Society Press Monograph, pages 50-51, which isincorporated herein by reference. A further discussion of spatialtransforms can found in Digital Image Processing, 2nd edition, Refael C.Gonzalez and Paul Wintz, pages 246-251, which is also incorporatedherein by reference.

It is contemplated that the dot pattern need not be a regular lattice ofpoints but may be derived by extracting stochastically reliable anchorpoints from snapshots of the incoming video stream captured in frame orimage memory. These may be further correlated using auto and crosscorrelation algorithms, Bissels algorithm for example, which assimilatescommon points from a cloud of points viewed from different locations.

FIG. 12 is a diagram showing the illustrative pattern of FIG. 11displayed on two adjacent and overlapping tiles. A first tile is shownat 290 and a second tile is shown at 292. The first tile 290 and thesecond tile 292 are overlapping by a predetermined amount, as shown at294. Each tile has a projector (not shown) for projecting a discreteimage onto the corresponding tile. In the embodiment shown, each of theprojectors is projecting a 9×9 array of dots. If the projectors wereproperly aligned, and there was no distortion in the system, each of thedots in the overlap region 294 would overlap one another. However, andas shown in FIG. 12, if the projectors are not aligned the dots do notoverlap one another.

To correct for this misalignment/distortion as, detected by the cameraand feedback system described herein, the present invention contemplatespre-warping the input video signal so that the corresponding dotsproperly align with one another. For example, the first projector, whichcorresponds to the first tile 290, projects dot 296, and a secondprojector, which corresponds to the second tile 292, projects acorresponding dot 298. A first transformation function may be providedfor effectively moving the location of the first dot 296 toward thesecond dot 298 when applied to the input signal of the first projector.Alternatively, or in addition to, a second transformation may beprovided for effectively moving the location of the second dot 298toward the first dot 296 when applied to the input signal of the secondprojector. If done properly, the first dot 296 and the second dot 298overlap one another on the screen. Further, if done in accordance withthe absolute or relative methods referred to above, then the compensatedimage is constrained globally and appropriately over the entire image.

Using a relative compensation method, it is contemplated that the firsttransformation function may move the location of the first dot 296toward the second dot 298 by an amount substantially equal to one-halfthe distance between the first and second dots. Likewise, the secondtransformation function may move the location of the second dot 298toward the first dot 296 by an amount substantially equal to one-halfthe distance between the first and second dots. This is a straightaveraging approach, devoid of global fit constraints like having toensure that the implied grid lines are straight through the firstderivative and equally spaced and is shown explicitly in FIG. 13.

Alternatively, or in addition to, the first transformation function maymove the location of the first dot 296 toward the second dot 298 by anamount that is weighted by a predetermined blending function or someother factor at the first dot 296 relative to the second dot 298, andthe second transformation function may move the location of the seconddot 298 toward the first dot 296 by an amount that is weighted by apredetermined blending function or some other factor at the second dotrelative to the first dot 296. This is a weighted average approach, andis shown explicitly shown in FIG. 14. Preferably, the weighting functionrelates to the blending function used for blending the color informationof the tiles. This may be a ramp or spline or some other suitablefunction known in the art.

When more than two corresponding dots must be considered, such as whenthree or more images overlap in a selected region, each of thecorresponding dots may be moved toward a corrected location. This may beaccomplished by using a similar averaging or weighted averagingapproach, as discussed above.

Other approaches are also contemplated. For example, it is contemplatedthat the transformation functions may maintain a predeterminedrelationship between selected dots. For example, dots 300, 302, 304 and306 are from a common row of dots, and thus should fall along a commonline 308. The transformation functions may maintain a linearrelationship between these dots while still compensating for thedistortion in the system. Likewise, dots 310, 312 and 314 are from acommon column of dots, and thus should fall along a common line 316. Thetransformation functions may maintain a linear relationship betweenthese dots while still compensating for the distortion in the system.Preferably, the linear relationship will provide continuity through thefirst derivative of the line functions and will preserve relativelyuniform spacing among the implied connecting lines.

In accordance with the above, FIG. 15 is a flow diagram showing anillustrative method for at least partially removing a spatial distortionfrom the display. The algorithm is entered at element 330, whereincontrol is passed to element 332. Element 332 causes at least one of theprojectors to project a discrete image that has a predetermined patternwith a number of features. Control is then passed to element 334.Element 334 captures a capture image of a selected portion of thecomposite image. Control is then passed to element 336. Element 336identifies a spatial distortion in the capture image by examining therelative location of selected features in the capture image. Control isthen passed to element 338. Element 338 determines a transformationfunction that will at least partially remove the spatial distortion fromthe composite image. Control is then passed to element 340, wherein thealgorithm is existed. Preferably, this method identifies the spatialdistortion of the display by comparing projected image of a tilerelative to the projected image of a neighbor tile, rather than or inaddition to being relative to a physical template.

FIG. 16 is a flow diagram showing a method for identifying atransformation for a tiled display to at least partially removing aspatial distortion from the tiled display. The algorithm is entered atelement 350, wherein control is passed to element 352. Element 352identifies a first feature in a selected overlapping region, wherein thefirst feature is projected by a first projector. This first feature maybe extracted from a snapshot of the incoming video image. Control isthen passed to element 354. Element 354 identifies a second feature inthe selected overlapping region, wherein the second feature is projectedby a second projector, and wherein the second feature corresponds to thefirst feature. Again, the second feature may be extracted from theincoming standard video input. Control is then passed to element 356.Element 356 determines the spatial relationship between the first andsecond features, illustrative of establishing the relationship among anensemble of features. Control is then passed to element 358. Element 358identifies a first transformation function for the first projector. Thefirst transformation function effectively moves the location of thefirst feature toward a corrective location when it is applied to theinput signal of the first projector. Control is then passed to element360. Element 360 identifies a second transformation function for thesecond projector. The second transformation function, appliedsimultaneously with the first described, effectively moves the locationof the second feature toward the corrective location when it is appliedto the input signal of the second projector. Control is then passed toelement 362, wherein the algorithm is existed. The calculation of thecorrection function may be done periodically while the output is passedto the transformation function for real-time correction.

In accordance with the present invention, the location of the dots maybe determined by: subtracting a black capture image from the captureimage that includes the dots; examining the resulting image contentabove a noise threshold using spatial filters which have an all-passkernel; measuring the center of gravity of the dots to find thecorresponding dot locations; eliminating dots whose energy threshold isbelow the threshold; sorting the dot locations for correlation withknown or expected dot patterns and deriving corrective transformationfunctions therefrom.

FIG. 17 is a graph showing the luminance domes for one LCD projectorwith various input intensities. As is evident, the magnitude of theluminance domes tends to increase as the input intensity increases.Also, the random brightness variation (e.g. noise) across the displaytends to increases as the input intensity increases. This variation isattenuated and thus the signal to noise ratio augmented by filteringmultiple time samples of dome capture images. This same general patternis found for red, green and blue. In addition, each color typically hasa different brightness value for the same input intensity. Further, inthe case of polysilicon LCDs for example, the shape of the patternchanges as a function of the input intensity level, requiring thecompensation function to attend to geometric and input intensityvariables.

To determine the color and luminance distortion of a projection system,and in particular, a tiled display system, direct view or projection, anumber of input signals of varying input intensity may be sequentiallyinput to the projection display. This may be done at initial calibrationor periodically. Each input signal may correspond to a flat field imageof a selected color. For example, a first input signal may correspond toa red flat field image having an LCD intensity of “255”. The next inputsignal may also correspond to a red flat field image, but may have a LCDintensity of “220”. Input signals having progressively lower intensitymay be provided until the input signal has a LCD intensity of “0”. Thisprocess may be repeated for both blue and green or other color flatfield images. A camera device may capture each of the flat field images,either as a single image if the field-of-view of the camera devicecorresponds to the entire display, or as multiple images if the cameradevice has a smaller field-of-view. The resolution of the camera devicemay be chosen to be appropriate with the selected field-of-view. Forexample, when the field-of-view of the camera devices is relativelywide, capturing an image of the entire display, a higher resolutioncamera device may be used. Likewise, when the field-of-view of thecamera device is relatively narrow, capturing an image of only a smallportion of the display, a lower resolution camera device may be used. Inany case, the resulting images are preferably stored as an array ofcapture images or reduced resolution capture images or a s compressioncoefficient capture images. Once collected, the non-desirablecharacteristics of each capture image can be determined including theluminance or color domes for each projector.

Once the luminance or color domes are identified, a ceiling and floorfunction which may be a linear or a spline or other suitable functionare preferably determined for both color (including hue) and intensity,across the entire display. For example, one projector may be brighterthan another at maximum of intensity (e.g. LCD “255”), and thebrightness provided by each projector may decrease near the edges of theimage. Accordingly, a ceiling may be selected to match the dimmestsuperposition area of the tiles when all projectors are operated atmaximum intensity. Likewise, a floor may be selected to match thebrightest super-position result of the tiles when all projectors areoperated at minimum intensity (LCD “0”).

Thereafter, a transformation function may be determined for compensatingthe luminance domes across selected tiles, and matching the brightnessand color of each tile with adjacent tiles, thereby resulting in alinear display system. For example, the transformation function may berepresented by a color look up table of captured or compressed colordomes, a nearest neighbor detection and identification function and aninterpolation function among the nearest neighbors to determine theinput level needed at the display to output the desired linear outputlevel. Preferably, the transformation function makes the luminancevariation across the entire display less than about two percent for flatfield test images, for example, which is less than onejust-noticeable-difference (JND) according to Weber's Law. To helpachieve this level of luminance uniformity, the transformation functionis preferably a function of the X and Y location on the tile and of theinput intensity level.

FIG. 18 is a schematic diagram showing the luminance domes forthree-tiled LCD projectors each at various input intensities. A firsttile 370, second tile 372 and third tile 374 each have different maximumbrightness values for a common LCD input intensity, such as an inputintensity of “255” as shown at 376, 378 and 380, respectively.

The transformation function for each of the tiles preferably compensatesthe luminance domes across selected tiles using the reduced luminancedome information to linearize the dome (x, y and input intensitydependent) behavior of each tile in the system and using the blendingfunction to allow for display tile superposition. The transformationfunction also matches the hue of each tile with adjacent tiles, usingthe feedback image information in combination with known tri-stimuluscolor transformations or equivalent functions. For example, thetransformation function for the first tile 370 may change the brightnessin accordance with feedback behavior acquired at some time from thecapture image of the first projector to lie along line 382 when theinput intensity to the first projector has a value of “255” and whensuperposed over the output of tile 372. In the diagram shown, this mayrequire that the center portion of the image be reduced more than theedge portions of the image because of the dome shape of the luminanceprofile. Likewise, the transformation function for the second tile 372may change the brightness in accordance with feedback behavior acquiredat some time from the capture image of the second projector to also liealong line 382 when the input intensity to the second projector has avalue of “255” and when superposed with neighbor tile outputs. Finally,the transformation function for the third tile 374 may change thebrightness of the third projector to lie along line 382 when the inputintensity to the third projector has a value of “255” and in accordancewith superposition with neighbor tile 372.

As alluded to above, the transformation functions are also preferablydependent on the input intensity that is provided to the projectors.This is the result of the dependence of the capture image domes on theinput intensity to image sources like the polysilicon LCD.

For lower input intensities, the transformation functions may change thebrightness of the first, second and third projectors to lie along, forexample, lines 386 or 388 as a function of the blending functions, theluminance dome dependence on the X, Y location on the screen and inputintensity to the tiled display system. In this way, the transformationfunctions may make the luminance, hue, and saturation variation acrossthe entire display relatively small regardless of the input intensityprovided. That accomplished or enabled, the image content may bearbitrary making the tiled display with camera feedback suitable fordisplaying general imagery.

In accordance with the above, FIG. 19 is a flow diagram showing a methodfor at least partially removing a luminance distortion from the display.The algorithm is entered at element 400, wherein control is passed toelement 402. Element 402 sequentially inputs one or more input signalsthat correspond to a flat field image of varying intensity to eachprojector. Control is then passed to element 404. Element 404 captures acapture image of selected flat field images. Control is then passed toelement 406. Element 406 identifies a luminance dome on one or more ofthe capture images. Control is then passed to element 408. Element 408determines a feedback transformation function (dependent on X, Y and/orinput intensity) that will at least partially remove the luminance domesfrom the composite image. Control is then passed to element 410, whereinthe algorithm is existed.

Finally, a method for determining the boundaries of each tile of a tileddisplay when using a camera that has a field-of-view that encompassesmore than one tile is contemplated. This method includes displaying awhite field image, for example, on all but a selected tile. The cameramay then capture an image of display including the selected tile. Thenit is a relatively simple matter to determine the boundaries of theselected tile by identifying the location where the white field imagebegins/stops. Another method is to display a checkerboard patternwherein each of the tiles assumes one of two flat field images. In thisembodiment, the boundaries for each tile may be determined byidentifying the location where each of the flat field imagesbegins/stops. Another method is to display a pattern of dots whose outerboundaries when detected by the camera in combination with a detectionfunction define the boundaries of each tile as well. These may be usedby the feedback processor-camera system to identify, among other things,the alignment of the projectors relative to one another. Further, andwhen the discrete images overlap one another, these methods Furtheridentify the extent of the overlap.

Having thus described the preferred embodiments of the presentinvention, those of skill in the art will readily appreciate that theteachings found herein may be applied to yet other embodiments withinthe scope of the claims hereto attached.

What is claimed is:
 1. In a system for controlling a tiled displaysystem that receives an input video signal, the tiled display systemhaving two or more displays, each manifesting one of a number ofdiscrete images separately onto a viewing surface or screen to form acomposite image, at least one of the discrete images overlapping anadjacent discrete image to form at least one overlapping region, theimprovement comprising: camera means for providing a capture image of atleast a portion of the composite image including at least a portion ofat least one overlapping region; determining means for determining ifthe capture image has one or more non-desirable characteristics; andidentifying means for identifying a transformation function that can beused to process the input video signal and provide processed input videosignals to selected displays to reduce the one or more non-desirablecharacteristics.
 2. A tiled display system according to claim 1, whereinthe two or more displays are projection type displays.
 3. A tileddisplay system according to claim 1, wherein the two or more displaysare front projection displays.
 4. A tiled display system according toclaim 1, wherein the two or more displays are rear projection displays.5. A tiled display system according to claim 1, wherein the one or morenon-desirable characteristics include spatial non-uniformity.
 6. A tileddisplay system according to claim 1, wherein the one or morenon-desirable characteristics includes color non-uniformity.
 7. A tileddisplay system according to claim 1, wherein the one or morenon-desirable characteristics includes luminance non-uniformity.
 8. Atiled display system according to claim 1, wherein the camera means is acolorimeter.
 9. A tiled display system according to claim 8, wherein thecalorimeter comprises one or more video cameras.
 10. A tiled displaysystem according to claim 1, wherein the tiled display system has aviewing side with said camera means located on the viewing side.
 11. Ina system for controlling a tiled display system that receives an inputvideo signal, the tiled display system having two or more displays, eachmanifesting one of a number of discrete images separately onto a viewingsurface or screen to form a composite image, the improvement comprising:camera means for providing a capture image of at least a portion of thecomposite image, the tiled display system having a viewing side withsaid camera means located on the opposite of the viewing side;determining means for determining if the capture image has one or morenon-desirable characteristics; and identifying means for identifying atransformation function that can be used to process the input videosignal and provide processed input video signals to selected displays toreduce the one or more non-desirable characteristics.
 12. A tileddisplay system according to claim 11, wherein said camera meansperiodically captures a new capture image during normal functionaloperation of the tiled display system.
 13. A system according to claim12, wherein said determining means periodically determines if the newcapture image his one or more non-desirable characteristics, and saididentifying means periodically identifies a new transformation functionthat can be used to process the input video signal and provide processedinput video signals to selected displays to reduce the one or morenon-desirable characteristics.
 14. A system according to claim 11,wherein said camera means captures the capture image during acalibration period.
 15. A tiled display system according to claim 11,wherein the camera means captures a portion of the composite image thatsubstantially corresponds to one of the number of discrete images.
 16. Asystem according to claim 11, wherein the camera means sequentiallycaptures a portion of the composite image, wherein each sequentiallycaptured portion substantially corresponds to one of the number ofdiscrete images.
 17. A method for controlling a display system that hastwo or more displays, each display manifesting one of a number ofdiscrete images separately onto a viewing surface or screen to form acomposite image, the method comprising the steps of: capturing a captureimage of at least a portion of the composite image, the capture imagebeing captured from a side of the viewing surface or screen that isopposite to a viewing side; determining if the capture image has one ormore non-desirable characteristics; and identifying a transformationfunction that can be used to process an input video signal and provide aprocessed input video signal to selected displays to reduce the one ormore non-desirable characteristics.
 18. A method according to claim 17,wherein said capturing, determining and identifying steps are repeatedperiodically during normal functional operation of the projectiondisplay.
 19. A method according to claim 17, wherein said determiningstep compares the capture image with a predetermined data set todetermine if the capture image has one or more non-desirablecharacteristics.
 20. A method according to claim 17, wherein saidtransformation function is determined during a calibration procedure.21. A method according to claim 20, wherein said calibration procedureincludes the steps of: sequentially inputting one or more input signalsthat correspond to a flat field image of varying intensities; capturinga capture image of selected flat images; identifying a luminance dome inthe capture images; and determining a transformation function that willat least partially remove the luminance domes from the composite image.22. A method according to claim 21, wherein the transformation functionis represented by a color look up table of captured color domes, anearest neighbor detection and identification block and an interpolationblock among the nearest neighbors to determine the input level needed atthe display to output the desired linear output level.
 23. A methodaccording to claim 22, wherein the transformation function isrepresented, at least in part, by a number of coefficients.
 24. A methodaccording to claim 22, wherein the transformation function is a bilineartransformation function.
 25. A method for controlling a display systemthat has two or more displays, each display manifesting one of a numberof discrete images separately onto a viewing surface or screen to form acomposite image, the method comprising the steps of: causing the atleast one of the two or more displays to manifest a discrete image thathas a predetermined pattern with a number of features; capturing acapture image of a selected portion of the composite image; identifyinga spatial distortion in the capture image by examining the relativelocation of selected features in the capture image; and determining atransformation function that will at least partially remove the spatialdistortion from the composite image.
 26. A method according to claim 25,wherein said predetermined pattern includes a dot pattern.
 27. A methodaccording to claim 26, further comprising the steps of: locating thedots in the capture image; and identifying a transformation functionthat will at least partially remove the spatial distortion from thecomposite image by comparing the location of the dots in the captureimage relative to an expected location.
 28. A method according to claim27, wherein said locating step includes the steps of: calculating thedot locations by subtracting a black capture image from the captureimage that includes the dots; examining the resulting image contentabove a noise threshold using spatial filters which have an all-passkernel; measuring the center of gravity of the dots to find thecorresponding dot locations; eliminating dots whose energy threshold isbelow the threshold; sorting the dot locations for correlation withknown or expected dot patterns and deriving corrective transformationfunctions therefrom.
 29. A method for calibrating a tiled displaysystem, the tiled display system having two or more displays, eachreceiving an input signal and each manifesting one of a number ofdiscrete images separately onto a viewing surface or screen to form acomposite image, the method comprising the steps of: providing an inputsignal to selected projectors to project a number of discrete images,each discrete image having a predetermined pattern with a number ofcorresponding features; capturing a capture image of at least a portionof the screen using a camera device; determining a distortion in thecomposite image by isolating the location of selected features in thecapture image; identifying a transformation function that can be appliedto the input signal of selected displays to at least partially removethe distortion from the composite image.
 30. A method according to claim29, wherein each display projects one of the number of discrete imagesseparately onto a screen in an overlapping manner to form a compositeimage having one or more overlapping regions.
 31. A method according toclaim 30, wherein at least one of the overlapping regions corresponds tomore than 25% of the corresponding discrete images.
 32. A methodaccording to claim 30, wherein at least one of the overlapping regionscorresponds to more than 50% of the corresponding discrete images.
 33. Amethod according to claim 30, wherein said determining step includes thesteps of: identifying a selected feature in a selected overlappingregion, wherein the selected feature is manifested by a selecteddisplay; identifying a selected feature in the selected overlappingregion, wherein the selected feature is correlated to a correspondingfeature; determining the spatial relationship among the selectedfeatures; and identifying one or more transformation functions for thedisplay, said transformation functions effectively moving the locationof the selected features on the viewing surface or screen toward acorrective location when applied to the input signal of the selecteddisplay.
 34. A method according to claim 33, wherein said firsttransformation function effectively moves the location of the selectedfeatures toward the corrective location by an amount that is weighted bya blending function of the tile exhibiting the selected feature.
 35. Amethod according to claim 33, wherein the predetermined pattern has apredetermined linear relationship among selected features.
 36. A methodaccording to claim 35, wherein said first transformation functioneffectively moves the location of the selected feature toward thecorresponding feature, and said second transformation functioneffectively moves the location of the corresponding feature toward theselected feature, while maintaining the predetermined linearrelationship between the features.
 37. A method for calibrating a tiledprojection display, the tiled projection display having two or moreprojectors, each receiving an input signal and each projecting one of anumber of discrete images separately onto a screen to form a compositeimage, the method comprising the steps of: sequentially inputting toselected projectors an input signal that correspond to a flat fieldimage of a first color for each of a number of luminance intensities;capturing a capture image of selected flat field images, the captureimage being captured from a side of the screen that is opposite to aviewing side; determining a distortion in the projection display byexamining the captured flat field images; and identifying atransformation function that can be applied to the input signal thatwill at least partially remove the distortion from the composite image.38. A method according to claim 37, where in the inputting, capturing,determining and identifying steps are repeated for each oil two or morecolors.
 39. A method according to claim 37, wherein said projectiondisplay includes a number of pixels, and said determining stepdetermines a distortion using more than one of the pixels.
 40. A methodaccording to claim 39, wherein said determining step determines anX-distortion by examining more than one pixel in an X-direction, and aY-distortion by examining more than one pixel in a Y-direction, so thatan X-Y dependence of the distortion can be determined.
 41. A methodaccording to claim 40, wherein said identifying step identifies thetransformation function by examining the X-Y dependence of thedistortion determined by the determining step.
 42. A method forcalibrating a tiled projection display, the tiled projection displayhaving two or more projectors, each receiving an input signal and eachprojecting one of a number of discrete images separately onto a screento form a composite image, the method comprising the steps of: providinga physical template that includes a predetermined pattern adjacent thescreen; capturing a capture image of at least a portion of the physicaltemplate using a camera device; and determining a camera distortion thatis introduced by the camera device by comparing the capture image with apredetermined expectation.
 43. A method according to claim 42, furthercomprising the steps of: removing the physical template; providing aninput signal to selected projectors to project a number of discreteimages, each exhibiting a predetermined pattern; capturing a captureimage of at least a portion of the screen using a camera device; anddetermining a projection system distortion that is introduced by theprojection system by comparing the capture image with a predeterminedexpectation.
 44. A method according to claim 43, further comprising thestep of removing the camera distortion before determining the projectionsystem distortion.
 45. A method for controlling a display system thathas two or more displays, each display manifesting one of a number ofdiscrete images separately onto a viewing surface or screen to form acomposite image, at least one of the discrete images overlapping anadjacent discrete image to form at least one overlapping region, themethod comprising the steps of: capturing a capture image of at least aportion of the composite image including at least a portion of one ormore overlapping region; determining if the capture image has one ormore non-desirable characteristics; and identifying a transformationfunction that can be used to process an input video signal and provide aprocessed input video signal to selected displays to reduce the one ormore non-desirable characteristics.
 46. A method for calibrating a tiledprojection display, the tiled projection display having two or moreprojectors, each receiving an input signal and each projecting one of anumber of discrete images separately onto a screen to form a compositeimage, at least one of the discrete images overlapping an adjacentdiscrete image to form at least one overlapping region, the methodcomprising the steps of: sequentially inputting to selected projectorsan input signal that correspond to a flat field image of a first colorfor each of a number of luminance intensities; capturing a capture imageof selected flat field images including at least a portion of one ormore overlapping region; determining a distortion in the projectiondisplay by examining the captured flat field images; and identifying atransformation function that can be applied to the input signal thatwill at least partially remove the distortion from the composite image.